Modules, Packages, and Libraries
================================

Python allows you to organize code into modules and packages. Using this
system, we can share and use each other's code.

In this video, I'm going to cover virtual environments and the import
statement. I'll also give some very basic directions on how to create your own
packages.

Virtualenv
----------

If you plan on importing anything, let alone installing anything, you should
create a virtualenv.

A virtualenv (venv for short) will isolate your packages and modules from the
system packages and modules. What someone else installs in their virtualenv
will not affect yours. And you can have different packages than the system
packages.

Creating a virtualenv is ridiculously easy.

.. code:: bash

  $ python -m venv <path>

``<path>`` is the path you want to create your virtualenv inside. Typically, I
name it ``venv`` and I put it at the top-level of whatever project I am
working on. For instance:

.. code:: bash

  $ cd Projects/calculator
  $ python -m venv venv

Note that you should be invoking venv with your Python 3.7 installation.

Once you have created your virtualenv, it's time to activate it.

.. code:: bash

  $ . venv/bin/activate # Linux/MacOS
  $ venv/Scripts/activate # Windows

Once you've activated your virtualenv, all the packages you install with
``pip`` will be installed in your virtualenv and be immediately accessible.
See https://PyPI.org/ for a list of publicly available packages. You may also
find packages you want to install at your favorite project's website.

import statement
----------------

The simplest form of the import statement is:

.. code:: python

  import <identifier>

This will import the module named <identifier> and assign it to a variable
of the same name.

As an example:

.. code:: python

  import sys

The exact steps Python follows are:

- Check ``sys.modules`` to see if the module has already been loaded. If so,
  just assign the module to a variable of the same name in your namespace.

- If the module has not been loaded, search for a file named ``<identifier>.py`` or
  ``<identifier>/__init__.py`` in the virtualenv. (Note: This is not the whole
  story. It's quite a bit more complicated than this.)

- Once the file is found, run it in its own namespace. (The module's global
  namespace is not the same as your global namespace. You can access its
  global namespace through the module attributes.)

- Registers the module in ``sys.modules`` and assign the module to a variable
  of the same name in your namespace.

.. note::

  You can import at any time. People typically like to import before running
  code, but this is not necessary.

  If you execute the import statement in a function call, then a local
  variable will be used for the module name, rather than a global.

Accessing the Contents of a Module
----------------------------------

Once you've imported a module, you can access objects within its namespace via
the attribute access ``.`` syntax:

.. code:: python

  import sys
  sys.modules

You should read the documentation on the module if you want to know what is in
it.

If you don't have the documentation available, or you just want a quick
reminder, then use ``help()`` or ``dir()``

.. code:: python

  >>> import sys
  >>> help(sys) # Documentation of all the objects
  >>> dir(sys)  # List of all the objects

ImportError
-----------

If anything goes wrong during the import process, an ``ImportError`` is
raised. You can catch this error and handle it:

.. code:: python

  try:
      import foo
  except ImportError:
      print("Error!")

Typically, ``ImportError`` isn't caught and handled. The only case where I
have done this is when I have code that needs to run in Python 2 and 3, and
the module is named differently.

ImportErrors could arise for any one of these problems:

- There is no module with that name.
- The module has a SyntaxError that prevents Python from compiling and running
  it.
- While running the module, an exception was raised.

There isn't much you can do except resolve the underlying problems. Maybe you
misspelled the module name, or you forgot to install it with ``pip``, or you
wrote the module yourself and created a SyntaxError or some other bug in it.

Importing multiple modules
--------------------------

You can import multiple modules at the same time:

.. code:: python

  import os, sys

This is the same as importing each of the modules one at a time.

.. code:: python

  import os
  import sys

Importing Modules in Packages
-----------------------------

Some modules are organized in a package.

.. note::

  - *Modules* are files in Python.
  - *Packages* are paths to the files in Python.

  For instance, ``foo/bar/baz.py`` would be the module ``foo.bar.baz`` and the
  package ``foo.bar``.

  This is an important distinction in Python, as we'll see when we write our
  own packages and modules.

.. code:: python

  import <package>.<module>

This will assign the top-level package to a variable of the same name. You can
access the module with the ``.`` syntax.

You may have packages nested in packages nested in packages. Use as many ``.``
as needed.

There aren't many nested packages in the Python library. Here's an example,
though:

.. code:: python

  import os.path

  os.path.join(...)

Importing with a different name
-------------------------------

Sometimes you want to import a module, but store it under a different name.

.. code:: python

  import sys as s
  s.modules

This will import sys but assign it to ``s``.

This works for package imports as well:

.. code:: python

  import os.path as p
  p.join

The benefit of this is the module, not the package, is assigned to the
identifier.

Importing specific objects
--------------------------

You can import individual objects from a module:

.. code:: python

  from <module> import <id>, <id>, <id>

If you have a lot of things, using parentheses is OK:

.. code:: python

  from <module> import (
    <id>,
    <id>,
    <id>,
  )
 
Note that you can also import from modules in a package this way.

This will import the module, and then assign the objects to variables of the
same name.

If you want a different name, use ``as``:

.. code:: python

  from <module> import (
    x as a,
    y as b,
    z as c,
  )

Module Special Attributes
-------------------------

Modules have a few special attributes.

- ``__name__``: The name of the module.
- ``__package__``: The name of the package of the module.
- ``__file__``: The location of the file the module came from.
- ``__doc__``: The doc string.

Running a module as a script
----------------------------

You can invoke any module as a script from the commandline:

.. code:: bash

  $ python -m <module name> <script parameters>

We did this with the ``venv`` module when creating a virtual environment:

.. code:: bash

  $ python -m venv venv

``if __name__ == '__main__':``
------------------------------

Sometimes you want to write a module that also functions as a script.

When a python file is loaded as a script, Python treats it as a module named
``__main__``. You can check the module's ``__name__`` from within the module
itself.

This means you can hide the code that should only be run as a script behind
this if statement:

.. code:: python

  ... module code ...

  if __name__ == '__main__':
      ... script code ...

There is a better way to do this using ``setuptools`` that I'll cover in a
later video.

Creating your own Packages and Modules
--------------------------------------

Creating your own Packages and Modules is a fairly involved process that
involves writing a ``setup.py`` file. There isn't a lot of consensus in the
Python community how to do this properly, so there is a lot of confusion
around it.

I recommend something simple and robust.

.. code::

  mycode/ # Project directory

    setup.py

    venv/ # For your virtual environment

    mycode/  # If a package

      __init__.py  # Empty

      mycode.py

      ...

    mycode.py # If a single module

To structure your code, first create a directory. You'll likely want to manage
this directory with ``git`` or a similar tool. (I'll create a video on what
you really need to know about git later.)

You need a package or module name. I'm using ``mycode`` as a stand-in for
that.

The package would live under a directory called ``mycode``. In that directory,
you need a ``__init__.py`` file, which I recommend leaving blank. If you have
any subdirectories, they each need their own ``__init__.py``, which I
recommend leaving blank. Any modules you want you just drop in the appropriate
directory with the appropriate name. Let's say you want a ``mycode.foo.bar``
module. In that case, create a directory called ``mycode/foo`` and drop an
``__init__.py`` there, then add in ``mycode/foo/bar.py`` for the module file.

If you just want a single module with no packages, then just create
``mycode.py`` in your project directory.

Next, you need a ``setup.py`` file. People try to get really fancy with their
``setup.py`` files, but I strongly discourage this.

Here's  bare minimum of what I would put in my ``setup.py`` file:

.. code:: python

  from setuptools import setup, find_packages

  setup(
      name='mycode',
      version='0.0.1',

      python_requires='>=3.7',

      # If a package, then...
      packages=find_packages(include=['mycode', 'mycode.*']),

      # If a module, then...
      py_modules=['mycode'],
  )

If you are serious about creating packages and modules, then I highly
recommend reading and understanding the content at
https://packaging.python.org/. I really can't do a better job than what they
have done.

That said, every project is different, and you're likely going to run into
cases where you're at a loss for how to properly package it. I can help. Find
me on discord.

Installing your Own Code for Testing
------------------------------------

Once you get your setup.py written, then install your code with ``pip install -e
<project directory>``. That's all you need. With the venv activated, you can
import your package or module however you see fit.

The ``-e`` option tells pip to not copy the files over, but to link them. This
means that when you change one of the files, those changes will appear when
someone imports your modules and packages again.

Note that if you add directories, you'll have to ``pip install`` again,
otherwise they won't be linked in to the venv. 

Distributing your Code
----------------------

Before you go launching your code to PyPI, talk to me first, please. I can
help you get your package production ready so that you won't make some rookie
mistakes when deploying packages.


Future Videos
-------------

- Going into setuptools.setup in depth
- Conflict resolution
- git